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ABSTRACT 



Personal Digital Assistants (PDAs) allow users a small 
profile, battery-operated, and relatively low-cost way to take many of the 
capabilities of a personal computer into the field. This combination of 
features suggests a great potential for the use of a PDA as a field data 
collection instrument. This research compared the use of one kind of PDA (the 
Pilot by U.S. Robotics) for this purpose. Seventy- five participants took part 
in a study comparing the usefulness of a paper-and-pencil data collection 
strategy, an optical mark recognition (OMR) form, and the use of a PDA. 
Results show that the paper-and-pencil method was the easiest and fastest to 
set up and get into the field, while the PDA method proved the fastest and 
most accurate for transferring collected data into a computer for later 
analysis. These results show that a PDA can be a useful tool for certain 
kinds of field-based social science research. An appendix contains the survey 
of attitudes about statistics used for data collection. (Contains 14 
references . ) (Author/SLD) 
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Abstract 

Personal Digital Assistants (PDAs) allow users a small profile, battery-operated, and 
relatively low cost way to take many of the capabilities of a personal computer into the 
field. This combination of features suggests a great potential for the use of a PDA as a 
field data collection instrument. This research compared the use of one kind of PDA 
(the Pilot by U.S. Robotics) for this purpose. Seventy-five participants took place in a 
study comparing the utility of a paper-and-pencil data collection strategy, an optical 
mark recognition (OMR) form, and the use of a PDA. Results show that the paper-and- 
pencil method was the easiest and fastest to set-up and get into the field, while the 
PDA method proved the fastest and most accurate for transferring collected data into a 
computer for later analysis. These results show that a PDA can be a useful tool for 
certain kinds of field-based, social science research. 
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Using a PDA for Field Data Collection 

New technologies have created an explosion in the number and kinds of 
computing devices. The most recent addition to the cornucopia of smaller and faster 
techno-wizardry is the Personal Digital Assistant, or PDA. Originally conceived of as 
just an electronic version of the pocket-sized, pen-and-paper date book, the latest 
devices boast features and capabilities not found on even the most advanced desktop 
computer systems. The best of these PDAs are small in size yet with large storage 
capacities, able to reliably recognize handwritten words (eliminating the need for bulky 
keyboards and mice), and with the ability to be quickly and easily synchronized with 
desktop computer databases. 

It would not be unusual, given these capabilities, for social science researchers 
to imagine these new PDAs as a potential new tool for data collection. PDA-based 
software does exist for this purpose, suggesting that development companies have 
been thinking along these lines. How well the use of a PDA compares to more 
traditional field-based methods remains unknown. The purpose of this study, therefore, 
is to investigate the potential utility of a PDA for field data collection, and to compare it 
to more traditional data gathering strategies. 

The PDA 



There is nothing new about using computer technology for data collection in 
field-based research. Small, handheld, general purpose computing devices have been 
in use since 1988 (Saudargas & Bunn, 1989). Research on these devices have shown 
their use to be as, if not more, effective than the use of other methods of data collection 
(Saudargas & Zanolli, 1990). More recent studies have expanded the notion of field- 
based research to include any activity when the researcher is away from his or her own 
office/laboratory, including wireless access to library based resources (Foster, 1995). 

A combination of factors seem to be driving this expansion in the use of 
handheld computers for research. Decreased cost, increased functionality, and a 
reduced size are all hallmarks of the last in technological advances. The most recent 
addition to the handheld, or PDA, family carry as much computing power as functional 
capacity as an entry-level desktop computer yet typically occupy less space than a VHS 
videotape (Lasky & McCracken, 1997). Although a relatively new device, manufacturers 
have been quick to fill the market with competing models, offering a dizzying array of 
features over a wide range of prices (Davids, 1996a and 1996b; Thompson, 1996; 
Marshall, 1996) 

One product that seems especially well suited to field research, due to its 
extreme small size and capable set of software, is U.S. Robotics’ PaImPilot product 
(Wayner, 1996; U.S. Robotics, 1996). This PDA has been described by one reviewer as 
"... the smallest, easiest-to-carry palmtop we tested — it’s about the size of a deck of 
playing cards and weighs less than 6 ounces, so it fits comfortably in my shirt pocket” 
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(Lasky & McCracken, 1997, p. 166). Powered by only two AAA batteries, the PaImPilot 
will run for two to three months of average daily usage before a battery change 
becomes necessary. Data loaded onto this device is easily transferred to a hosting 
computer by a “hot-link” cable and single button functionality (Byte Magazine, 1996). 
Recent upgrades to the Pilot PDA (Blodgett, 1997) provide a backlit screen, increased 
memory, expense tracking, and a modem with e-mail functionality. 

One of the most impressive features of the Pilot PDA is its ability to accept 
downloaded software. A number of third-party software developers have begun creating 
applications for the Pilot, including Pendragon Software’s PilotForms package. This 
program, occupying just over 75kb of memory on the Pilot (and approximately four to 
six megabytes on the accompanying personal computer) and costing under $50, 
provides user-programmable data entry and editing capabilities (Pendragon Software 
Corporation, 1997). Information entered into a PilotForms data entry screen on the Pilot 
is automatically transferred to a Microsoft Access database on the host personal 
computer when the unit is hot-linked. 

Can this technology really improve upon more tried-and-true methods used for 
collecting data in the field. In this study I attempted to ascertain the utility of using a 
PDA like the Pilot for field data collection, as compared to more traditional methods like 
paper-and-pencil recording and the use of an optical mark recognition (OMR) form. If 
the added expense and learning/programming of the PDA is to have a true utility for 
field-based researchers it must be demonstrated that collecting data using a PDA has 
significant benefits over existing, lower cost techniques. 

Methods 



Participants 

Seventy-five individuals from the campus community at a large, mid-western 
university volunteered to participate in the study. Subjects were chosen as a sample of 
convenience rather than for any determining characteristics. These 75 individuals were 
randomly distributed into three different groups of 25 persons each. One group received 
the survey using a paper-and-pencil methodology, another group using a optical mark 
recognition recording style, and the third using a PDA. In order not to bias the study 
participants were not told the true purpose of this research; rather, each was informed 
that the purpose of the study was to examine attitudes about research courses (the 
content of the questionnaire). 

Materials 



Three different forms of the survey instrument, expanded from a work by Wise 
(1995, p. 403), were created for this research (Appendix A). The first form of the 
instrument was utilized in the paper-and-pencil methodology. Multiple copies of this 
form were created, and respondents answers were recorded directly onto the form by 




5 



Using a PDA 5 



the researcher. The second form of the instrument was used in the OMR methodology. 
One copy of the instrument was made, with the researcher recording respondents 
answers onto an OMR sheet placed adjacent to the questionnaire. The final form of the 
instrument was entered into a PilotForms database and uploaded to a Pilot PDA. 
Participant responses were recorded directly into the Pilot - no paper form was used. 

Design and Procedure 

Three research assistant, blind to the true purpose of the study, were employed 
for the actual field data collection. Each assistant was given one of the three 
methodologies, access to a personal computer, and necessary documentation (for the 
OCR and PDA methods). All three assistants were trained in the administration of the 
survey, and were urged to complete each interview as rapidly as possible while 
recording all responses and other comments made by each participant. 

The field data collection proceeded in three distinct stages. In the first stage 
each assistant was to convert the survey items into a format appropriate for their 
methodology. In the case of the paper-and-pencil strategy this meant: typing the survey 
questions into a word processor, proofreading and printing the final survey, 
photocopying 25 duplicates of the survey, and locating a clipboard and pencils to use in 
the field. In the case of the OMR strategy this meant: typing the survey questions into a 
word processor (with item numbers for OMR form matching), proofreading and printing 
the final survey, acquiring blank OMR forms and blank paper (to record the open-ended 
responses and other comments), and setting up the OMR machine (a Scantron 8200) 
and software (Scantron’s Scanbook program) to record the scanned forms. In the case 
of the PDA strategy this meant: setting up the PilotForms software on the personal 
computer, typing the survey items into this software, downloading the completed form to 
the Pilot PDA, and practicing with the PDA to insure proper operation. 

The second stage involved the actual conduct of the survey with the participants. 
Twenty-five participants were randomly chosen for each methodology, for a total 
sample of 75 participants. 

The third stage involved transferring the data from the collection format into the 
computer for further analysis. The desired outcome from this stage were two computer 
data files: an SPSS (version 7.5 for Windows) systems file containing the coded 
categorical and other numeric response items, and a text file containing the participant 
responses from the open-ended questions, in addition to any other comments given by 
the participants. For the paper-and-pencil methodology this stage involved typing each 
of the completed field form.s into the appropriate software package, then checking each 
data entry for errors. The OMR strategy involved typing the textual responses into the 
computer, and using the Scantron 8200 to scan the coded responses. These scanned 
responses were automatically placed into delimited text file which was then imported 
into SPSS. The PDA methodology involved hot-linking the PDA to transfer the data 
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from the PDA into a Microsoft Access database on the personal computer. This file was 
then manipulated to export the coded data to SPSS and the textual data to a text file. 

The research assistants recorded the total time for each stage of the process, in 
addition to their own thoughts about their data collection methodology. Assistants were 
debriefed at the end of the study, and their information collected and tabulated. 

Results 



Stage One Results 

The paper-and-pencil strategy was the fastest and easiest to set up, taking only 
three-quarters of one hour to prepare. The OMR methodology required the next most 
amount of time, taking just over one-half hour to type the questionnaire and almost 75 
minutes to program, test, and debug the Scantron 8200 for the actual scanning. The 
PDA strategy took the longest to set up, requiring almost four hours of time. Much of 
this time was spent experimenting with different software options, and learning how to 
use the PilotForms program. The assistant working on this strategy did acknowledge, 
however, that future surveys using the PDA could be set up quite quickly, perhaps even 
as fast as the paper-and-pencil method, now that the critical how to was accomplished. 

Stage Two Results 

The mean interview time for all three methods was just over three minutes at 185 
seconds. A one-way analysis of variance was used to test whether one method was 
significantly faster than another. No significant differences (e > .05) were found 
between the paper-and-pencil method (M = 174 seconds), the OMR method (M = 201 
seconds), and the PDA method (M = 180 seconds). 

Stage Three Results 

The PDA methodology was the fastest and easiest of the three methods when it 
came to transferring the data into the computer. The hot-link process took a total of 12 
seconds, while formatting and exporting the data to the appropriate file formats took 
another four minutes. Since the PilotForms software will not produce a native SPSS file 
it was necessary to export the data first into an Excel spreadsheet file, then use SPSS 
to read this Excel file. An addition five minutes was spent spot-checking the final SPSS 
and text data files against the information stored in the PDA to insure accuracy. No 
errors were found in either the coded or textual data. 

The OMR methodology was the next fastest. While it required about 25 minutes 
to type the open-ended responses into the computer using a word processor (Microsoft 
Word), the actual scanning of the OMR forms was accomplished in just over two 
minutes. The Scanbook software produces a delimited text file as output. This file had 
to then be imported into SPSS, a process which took another 1 1 minutes of typing field 
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definitions. In a few cases (6 forms out of the 25 total) there were one or more errors in 
the scanned data. These errors were corrected directly in the SPSS spreadsheet editor 
by comparing the original OMR form to the scanned information. In all cases these 
errors were due to either too light a marking, or marking outside of the bubble 
boundaries, on the OMR form. It was not necessary to spot-check the entered data, as 
incorrect form reads showed up as specific error values in the SPSS spreadsheet. The 
process of locating these errors, and correcting them, took another eight minutes. 

The paper-and-pencil method was the slowest of the three methods, requiring 22 
minutes to transcribe the open-ended responses and another 28 minutes to transfer the 
coded responses into SPSS. Part of the time needed was used to create the field 
definitions in the SPSS spreadsheet, while the rest was spent entering the actual data. 
Unfortunately, no easy method existed for checking the accuracy of the transcribed 
work. A spot-check of five respondent’s data revealed one (or more) errors in 2 of the 5. 
This error rate was deemed unacceptable, and the entire SPSS data set was then 
cross-checked and corrected against the paper originals. The process took another 12 
minutes to accomplish. 

Only a limited examination was performed of the actual data collected, as the 
purpose of this study was to examine the methodology used and not the instrument 
results itself. Comparisons (using one-way analysis of variance and chi-squared 
statistics) were made, however, on each question across each of three methodologies 
to insure that no one method experienced a sample too different than any other. In no 
case were any statistically significant differences found on any item between the three 
groups. All three methods experienced very equal samples of respondents, even as it 
concerns the length - number of words used - in their open-ended responses. 

Conclusions 

The paper-and-pencil methodology was the easiest and fastest to get into the 
field, although it turned out to be the most time consuming and prone to errors of the 
three methods when it came to transcribing the data into the computer. The PDA 
method, on the other hand, was the fastest (and most accurate) of the three methods in 
moving its data from the field into the computer, although it required a substantial 
investment of time to learn the software and program the survey. The OMR strategy 
seemed to split the difference between the other two, requiring only a modest 
investment of time to set up but a somewhat longer period to transfer data. 

In a similar fashion the research assistants favored the paper-and-pencil method 
over the other two when it came to getting the instrument out into the field quickly. .All 
three were amazed at the speed and accuracy of the PDA on the third stage of the 
study, and felt that the PDA might have benefits in longer surveys, more complicated 
instruments, or those involving larger numbers of subjects. The PDA was also the 
smallest of the three field instruments used, making it the easiest to carry and the least 
obtrusive for the participants. 
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The results from this research suggest that a paper-and-pencil technique would 
be much more appropriate, and faster to implement, when a quick study using a limited 
number of items needed to be performed on a small sample. The OMR technology will 
likewise continue to experience for very large samples, and groups of participants that 
are not contacted in person (such as through the mail). The PDA, however, seems to 
be well suited to fill a niche where easy researcher mobility, a high degree of data 
transfer accuracy, and not too large sample is being utilized. Under these 
circumstances, especially when the researcher is comfortable with the software being 
used, the PDA is quite impressive with its potential to handle many different kinds of 
items, responses, and designs. 

The Pilot PDA, using the PilotForms software, seems like it would make a fine 
addition to a field researchers toolbox of techniques. Although not for everyone, a 
device like this could be used quite effectively for the researcher willing to invest the 
time to learn how to operate the software. Undoubtably future innovations in hardware 
and software will continue to improve on the portability, features, and utility of these 
personal digital assistants. Researchers involved in field studies requiring these 
capabilities should investigate the potential use of a PDA as one means of reducing 
their workload while improving data accuracy. 
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Appendix A 

Survey of Attitudes About Statistics Courses 

Thank you for participating in this survey of attitudes about statistics courses. 
This survey will only take a few minutes. 

[Record today’s date and time] 

Let’s begin with some basic questions about you. 

First, what is your gender? Male Female 

How old are you? years 

What is your ethnicity? White, non-Hispanic 

Black, non-Hispanic 
Hispanic 
Asian/Pacific Islander 
Native American/Alaskan 
Other 
No Answer 



What is your academic major? 

Have you ever taken a statistics course before? Yes No 

[If NO, skip the next three questions] 

The next questions are about the statistics course(s) you have taken. 

How many statistics courses have you taken? 

On a scale of 1 to 5, with 1 being worst and 5 being best, 

how much did you enjoy these courses overall? 1 2 3 4 5 

Would you recommend any of the statistics courses you 

took to another student? Yes No 



[continue on the back] 
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The next part of the survey asks your opinion about statistics courses in general. 
Answer these next questions on a 1 to 5 scale, with a 1 being “Strongly Disagree” 
and a 5 being “Strongly Agree”. Ready? 



The thought of being enrolled in a statistics course makes be nervous. 

Statistics seems very mysterious to me. 

I see being enrolled in a statistics course as a very unpleasant 
experience. 

I would like to continue my statistical training in an advanced course. 

I wish that I could have avoided taking my statistical course. 

I get upset at the thought of enrolling in another statistics course. 

I feel intimidated when I have to deal with mathematical formulas. 
Statistics is too complicated for me to use effectively. 



1 2 3 4 5 
1 2 3 4 5 

1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 



Would you share with me any other thoughts you might have about statistics courses? 



[Record today’s date and time] 

Thank you very much for participating. Have a nice day! 
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